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ABSTRACT 

More than half the minority group students who 
completed the 1988/89 Science and Technology Entry Program (STEP) at 
Manhattanville College (New York) scored in the 90-95 percentile on 
tests of critical thinking, suggesting ability to do well in 
mathematics, science, and other studies, STEP comprises the following 
goals: (1) enable students to enter and persist in college 
preparatory courses by teaching essential critical thinking, study, 
organizational, communication, scientific, and mathematical skills; 
(2) promote self-esteem and self-confidence; and (3) develop career 
awareness through exposure to minority role models. Fifty-two 
predominately black 7th-, 9th-, and llth-grade students completed the 
first year summer and academic components of the program. A variety 
of standardized tesLs and measures were used to evaluate student 
progress, including the Preliminary Scholastic Aptitude Test (PSAT) , 
the Rosenberg Self Esteem Scale, and the Ravens Coloured Progressive 
Matrices. Analysis of student scores indicates that their self-esteem 
was positive and their performance on standardized tests was 
acceptable- The discrepancies between scores on tests of critical 
thinking and scores on standardized tests suggest that the program 
should increase emphasis on critical thinking skills and align the 
content of mathematics and science modules more closely with those of 
the regular school curriculum. This ^udy is intended to become the 
first part of a 3-year longitudinal evaluation. Statistical data are 
included on eight tables. A list of 15 references is appended. 
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Abstract 

Tl*e Science and Technology Entry Program (STEP) was funded 
by the New York State Education Department at Manhattanville 
College in August 1988 to: 1) enable STEP students to enter and 
persist in college preparatory courses by providing them with 
essential critical thinking, study, organizational, 
communication, scientific, and mathematical skills necessary for 
academic success, 2) promote students self-esteem and self- 
confidence, and 3) expend student's career awareness. Fifty two 
7th, 9th, and 11th graders completed the first year summer and 
academic components of the program • This paper reports the 
results on students' critical thinking, self-esteem, and 
standardized scores after one year in the program. More than 
half of the STEP students scored in the 90-95 percentile on tests 
of critical thinking indicating likely ability to do well in 
math, science, and other studies. In general the self-esteem of 
STEP students was positive and their standardized test scores 
were acceptable. Tracking their achievement through the second 
year of the program, comparing the results of the first cohort of 
STEP students with the second cohort, and comparing their results 
with a matched group of non STEP students remain to be done. 
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Improving student achievemert and thus the chances of 
minority students to be admitted to the more select colleges has 
been a goal of numerous high school and college programs 
(Williams, 1987; Comer, 1987). Lower standardized scores, 
insufficient college preparation courses, and lack of a science 
and math background are but some of the reasons for colleges 
rejected minority students applicants. Closing this achievement 
gap has called for an accelerated approach to schooling (Levin, 
1987) . 

In this spirit, faculty members of Manhattanville College 
and Woodlands High School in the Greenburgh #7 School District 
worked together to develop and implement a program of study and 
skill development for minority youtn which focuses specifically 
on the sciences, mathematics, and technology. Manhattanville 
Collage possesses two critical elements required for a model 
Science and Technology Entry Program (STEP) program: an academic 
program that takes into accov . the needs of each individual and 
an impressive success rate with students from a wide range of 
educational and social backgrounds. Woodlands serves a diverse 
population of students. While many go on to college, there are a 



significant number of minority students at Woodlands who are not 
in academic programs or courses of study which will lead to 
careers in math science professions. The STEP program uses the 
College to augment the high school program thus providing STEP 
students with academic support throughout their secondary 
schooling, and with the skills, knowledge and attitudes necessary 
to pursue higher education and careers in mathematical, 
scientific and technological fields. 

The Manhattanville-Woodland STEP Program involves an 
intensive month of study at the College during the summer and 
weekly activities at the College during the academic year. 
Eleventh and twelfth grade students are involved in internships 
at various business and scientific sites throughout the 
Westchester area. In the first year, the program was designed to 
accommodate sixty students in three cohort groups: twenty 
seventh graders, twenty ninth graders, and twenty eleventh 
graders. In the second and subsequent years, the program will 
incorporate one hundred-twenty students. The aims of the 
Manhattanville-Woodlands STEP Program are to: 

1. Enable STEP students to enter and persist in college 
preparatory courses during high school by providing them 
with the essential critical thinking, study, organizational, 
communication, scientific and mathematical skills necessary 
for academic success. 

2. Promote STEP students 1 entry into higher education and 
careers in fields which involve the disciplines of 



mathematics and science through their participation in a 
systematic program of study in math and science. 

3. Promote STEP students' self-esteem and self-confidence 
through a series of academic and social activities, thus 
enabling them to succeed personally and professionally in 
school and careers. 

4 . Expand STEP students 1 career awareness through exposure to 
minority role models and internship and volunteer service 
experiences, thus broadening their career choices. 

5. Develop a model program comprised of instructional modules 
in the areas of math, science and technology, and counseling 
for skill development and career planning which can be 
replicated. 

The program is enhanced by the support of parents and 
students who participate in planning, educational and technical 
consultants, counseling and academic support staff from the high 
school and the college, and minority career mentors who work with 
small groups of students throughout their years in the program. 

The program is administered by Manhattanville College 
through the STEP program director, a person who combines 
expertise and knowledge in the fields of mathematics, science and 
technology, a:id a support staff. The program features an 
advisory board composed of college faculty and administrators, 
Greenburgh school district faculty and administrators, parents, 
and minority business leaders and educators who provide guidance. 
Staff development focusing on analytical thinking skills, 



cooperative learning strategies and multicultural education is 
provided for faculty and staff from both institutions. 

DESCRIPTION OF THE STUDY 

The assessment of the Manhattanville-Woodlands High School 
STEP Program in this study covers the period from August 1988 to 
June 1989. The information presented here is intended to become 
part of a longitudinal study and a detailed examination of the 
program over a three year period. 

Evaluation studies, such as this one, recognize that the 
program is dynamic, that is, that it is in operation, changeable 
and taking place in a naturalistic or field setting. While 
experimental rigor is applied when appropriate, it is necessary 
to augment the study with descriptions of contextual variables 
and to utilize the methodologies and perspectives of various 
disciplines to understand the processes and functioning of the 
program. 

The objectives of the study are to assess and monitor the 
progress and performance of STEP students with regard Al) 
critical thinking, A2) self-esteem, and A3) standardized tests. 

DESIGN 

Our approach is based on a pragmatic philosophy that results 
in a quasi-experimental design. The longitudinal nature of the 
program led to the ura of two analytical frameworks. The first 
relates to student variables; the second to program variables, 



thus providing a two-factor design in which Factor A describes 
the basic student objectives and Factor B describes the student 
population. (Fig. I) Bl indicates seventh grade students, B2 
ninth grade students and B3 eleventh grade students. 

FIG. I 

TWO FACTOR DESIGN FOR EVALUATION OF THE STEP PROGRAM 
Al A2 A3 
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Beginning with Year Two, there will be new students in the 
program who wore not participants during the program's first 
year. Thus, there wil3 eventually be students at each grade 
level 7 to 12. 

Comparison of their experience with that of the initial 
cadre of fifty-two students will allow for a more complex and 
richer analysis of first year data. In addition, it is planned 
to establish comparison groups of students who are not part of 
the STEP program. Establishing comparison groups with similar 
backgrounds to the STEP students, and with different backgrounds 
would allow us to more accurately determine the extent of the 
impact of the program. 

DATA COLLECTION 
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An overview of the measurement tools used to determine 
student outcomes expressed in the evaluation objectives is 
presented in Table I. Standardized tests such as CTBS and PSAT" 
taken from school records. 
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TABLE T 



STU nrNT OUTCOME MEASUREMENT TOOLS 



OBJECTIVES 



STANDARD 
TESTS 



STEP 
TEACHER 

assessment: 



ROSENBERG 

PROOKOVER 



/ 

FACULTY 
FEEDBACK 
WEEKLY 
REPORTS 



RAVELS 
MATRICES 



GRADE POINT 
AVERAGES 



X 



Comriunica 

-tion x 

Math & K >: 

Science 

. . • - • v X 

Scientiric 

Inquiry 

(Critical 

Thinking) 

Self Esteeir. 
Achievement 
Responsibil 
ity 



Career 
Awareness 

School X 
Attendance 



NOTES: 1. Standardized tests administered yearly by the school district CBTS and SAT 
will be used.^ teach . ers prR am pQst test will be administered at the beginning and end 
of each f a - e f f _^? g ^- cri ,, cal thinking measures will be administered yearly. 
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Two sets of instruments were used to acquire initial 
measures of student self-esteem and achievement responsibility. 
The first was a ten-item Rosenberg scale of self-esteem and the 
second was the Brookover eight-item measure of self-concept of 
academic ability. 
Self-Esteem 

The influence of self-concept on classroom behavior and 
performance has been of continual interest to educators and 
researchers for many years (Haggard 1981). Moreover, research 
findings leave little doubt that better students tend to have 
higher self -concept . Studies using a variety of measures of both 
scholastic competence and self-concept have demonstrated this 
relationship (Fine 1986). But while the positive relationship 
between level of self-concept and academic achievement has been 
well documented, the relationship between self-esteem and 
academic achievement has also been found across a wide variety of 
studies in which these terms, self -concept and self-esteem, have 
similar definitions. 

Generally, researchers theorize that students, regardless of 
their level and pattern of particular talents, should experience 
a school setting that will strengthen, not damage their self- 
concept and self-esteem. The school should help students at all 
levels of talent achieve feelings of success and of acceptance 
that are important in relating appropriately to others and in 
becoming successful members of society. 

One self-esteem scale that seemed to meet the demands of 
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this evaluation was constructed by Rosenberg (1965). His study 
presents the results of a large-scale survey focusing exclusively 
on the way adolescents see and feel about themselves. The survey 
deals with several dimensions of the self and devotes 
considerable attention to self-value. But the main concern, or 
the pivotal variable is self-esteem. In other words, what is the 
direction of the self-attitude? Does the, individual have a 
favorable or unfavorable opinion of himself? Does he consider 
himself worthy or unworthy? The main aim of the study was to 
specify the bearing of certain social factors on self-esteem and 
to indicate the influence of self-esteem on socially significant 
attitudes and behavior. 

Rosenberg selected items which openly and directly dealt 
with the dimension under consideration. Respondents were asked 
to strongly agree, agree, disagree, or strongly disagree with 
items such as: 

1. On the whole, I am satisfied with myself. 

2. At times I think I am no good at all. 

The full set of items is presented in the section on results. 

Jl Positive" and "negative" items were presented alternately 
to reduce the effect of respondent set. There is little doubt 
that the items generally deal with a favorable or unfavorable 
attitude toward oneself. The criteria of ease of administration, 
economy of time, unidimensionality , and face validity seem to 
meet both practical and theoretical demands. 
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Academic Ability 

Students' perceived self-concept of academic ability (SSCOA) 
was measured by an instrument developed by (Brookover, 1962). 

The SSCOA scale, designed by the Michigan State University 
research team under the leadership of Wilbur Brookover, measures 
the "self -concept of ability as a learner in school for academic 
subjects". As noted in the Michigan study, this was the first 
instrument developed to measure self -concept of academic aoility. 

Our modified version is a set of eight multiple choice 
questions designed to test the students 1 perception of a) their 
ability to achieve and b) the importance of grades. 

The format of the instrument is illustrated by two sample 
items: 

±. How do you rate yourself in school ability compared to your 

close friends? 

I am the I am I am I am I am the 

poorest below average above best 

average average 

2. What kind of grades do you think you are capable of getting. 

Don't care Mostly Mostly Mostly Mostly 
about any D's C's B's A ! s 

particular 
grade 

Students are asked to circle the answer which best answers each 
question. 

Critical Thinking 
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In recent literature, the term "critical thinking" has been 
a frequently discussed consistent topic in education circles, 
occurring with insistent regularity in the associated literature. 
Despite a general consensus of opinion that to think "critically" 
is a major constructive skill to be fostered and promoted in all 
students, such ostensible unity tends to crumble when "critical 
thinking" requires a definition. Therefore in order to ascertain 
clarity for the study we have provided a definition of "critical 
thinking". 

Critical thinking involves a certain judicious use of 
reflective skepticism. It does not take purported truth for 
granted. It considers alternative hypotheses and possibilities. 
"In part, critical thinking involves seeing when a certain common 
procedure is fruitless by entertaining alternatives to it". 
(McPeck 1981). In addition, critical thinking is productive; it 
seeks optimal solutions. Above all it involves knowing something 
about the field in question so as to trigger or produce "useful" 
questions, it cannot exist in isolation from experience, and 
therefore cannot justifiably be taught in isolation in the 
classroom. 

In light of this definition we may conclude that any general 
(ie., not subject specific) test of critical thinking must be a 
measure of both observation and clear thinking (cognitive 
processes) , and cannot simply be an instrument for testing 
general intelligence. 

The Raven's Progressive Matrices Scale (1938-1983), 
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developed in Britain by John C. Raven is designed to test 
critical thinking and cognitive ability, and is considered a 
relatively non-biased test of these skills. It is the quest for 
reduction of test bias when measuring cognitive abilities of 
school children, particularly in communities with ethnically 
diverse populations, or with large groups of economically 
disadvantaged and/or under represented peoples, which has 
prompted the search for "culture-fair" instruments. 

Raven's Colored Progressive Matrices (1947), in particular, 
are believed to be, if not completely "culture-fair", then 
certainly "culture-reduced", mainly because they correlate 
highest with WISC Block Design subtests and lowest with 
Information scores as found by Martin and Wiechers (1954). The 
value o~ such an instrument for use among the culturally-diverse 
school populations of many urban American schools is justifiably 
increased in view of the perspectives offered by the range of 
tests most frequently used to measure thinking skills. 

The two most commonly used tests for measuring critical 

thinking are the Watson-Glazer Critical Thinking Appraisal and 

the Corm 11 Critical Thinking Tests. John E. McPeck in his book 

"Critical Thinking and Education" (1981) argues that: 

neither test, in fact, measures critical 
thinking in any reasonable sense because, 
first, neither the tasks nor the results of 
these tests show any significant difference 
from those involved in "general intelligence" 
(that is, IQ) / the tests preclude the use of 
critical thinking in any defensible sense of 
that term. 
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On the other hand, McPeck does acknowledge that Cornell is 
an improvement over Watson-Glazer in two respects. First, the 
directions are clearer and straightforward, and the tasks 
involved are a direct manifestation of the instructions. Second, 
the authors recognize the difference between "self-contained" 
questions and "open" (or divergent) questions. 

Another widely-used test of cognitive ability is the Ross 
Test of Higher Cognitive Processes which purports to measure, 
among other skills, those of comprehension of relationships among 
ideas, recognition of logical fallacies in arguments and 
distinguishing a conclusion from supporting statements. 

Nonetheless, although the test's authors contend that it is 
not a measure of intelligence, due to the low correlation between 
the Ross test and the Lorge-Thorndike Intelligence test, there is 
little actual evidence to support their claim that the Ross Test 
measures higher level thinking. 

Evidence to support the case for increasing the use of 
Raven's Progressive Matrices comes from a variety of sources, not 
the least of which is the application of the Matrices scale in 
research requiring a "culture-fair" test of critical thinking. 

"The Matrices test devised by (Raven 1 s, 1966) is a means to 
assess a person's present ability to perceive and think clearly, 
irrespective of past experiences or present ability for verbal 
communication. They are a test of observation and clear 
thinking" . 

M. Maqsud (1983), in his study of the relationships of locus 
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of control to self-esteem, academic achievement, and predication 
of performance used the Raven's Standard Progressive Matrices 
(RSPM) as an instrument to measure his subjects 1 levels of 
cognitive ability because w there is evidence (Burke, 1958) that 
the RSPM is .least affected by cultural values". Maqsud found 
significant correlations between internal/external locus of 
control scores and RSPM scores among his Nigerian schoolboy 
subjects • 

The Raven's Colored Progressive Matrices (CPM) were 
administered to subjects of Sewell and Walker's (1982) study of 
the effects of material and symbolic incentives on the learning 
ability of low SES black children. The Raven's CPM was accepted 
by the examiners as a widely-used test of cognitive ability in 
low socioeconomic subjects. 

This latter assertion is not consistent with the findings of 
both the 1983 and 1979 U.K. standardizations, in which the test 
items scaled in the same way in all socio-economic groups. The 
wo- v of Hoffman (1983) suggests that the same is also true of 
different ethnic groups within the United States. 

"The differences in mean scores between children coming from 
different backgrounds (which are, in any case, minor compared 
with the variation between children from similar backgrounds) 
cannot, therefore, be explained by saying that the Tests are 
"foreign to the way of thought" of children from certain 
backgrounds, (Raven, 1985) . 

The Raven's Progressive Matrices are a widely accepted 
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"culturally-fair" test of cognitive ability, as opposed to a 
general intelligence test which is indubitably bound to 
discriminate between subjects 1 cultural and social backgrounds. 

The Matrices presented the students with 36 problems in the 
form of a pattern with a piece taken out cf it. The task is to 
select the missing piece from eight choices that are presented 
below the pattern (see figure II for sample problem) . The 
problems from 1 to 36 become progressively more difficult, hence 
the name progressive matrices. 

FIGURE II - SAMPLE PROBLEM: RAVEN'S PROGRESSIVE MATRICES 
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This test of observation and clear thinking is relatively easy to 
administer and score and is particularly useful for students 
wishing to pursue science or technical studies as long as the 
relative importance of other features is also considered, 

RESULTS 

Student Recruitment: Given the program' s late starting date due 
to late notification of the award, recruiting efforts were highly 
successful. The program was designed for sixty students. Eighty 
students applied and 52 were accepted in the program - 20 seventh 
graders, 16 ninth graders and 16 eleventh graders. The success 
of the recruitment effort was due in large part to the 
cooperation that exited between the Manhattanville Administration 
and the Woodlands High School staff. On the whole, enrollment 
remained steady throughout the year at around 52 students, (49 in 
the fall and 52 in the spring) . 

Sixty-one students participated in the STEP Program, 22 
eleventh graders, 19 ninth graders and 20 seventh graders. About 
sixty percent of the students were males (37 out of 61). All 
students were black, hispanic, or Asian; the vast majority were 
black (56 out of 6i) . They ranged in ages from 11-16 (Table II). 
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TABLE II 

STEP STUDENTS BY GRADE AND ETHNICITY 





NO. 


M 


F 


BLACKS 


HISPANIC 


ASIAN 


11 


22 


10 


11 


18 


3 


1 


9 


19 


5 


14 


19 


0 


0 


7 


20 


8 


12 


19 


1 


o 


TOTAL 


61 


24 


37 


56 ■ 


4 


1 



! 

The variance in attendance for eacn STEP period was due to ! 
some students dropping and others being added to the program. j 
Apparently enrollment also depends on other activities at the 

| 

high school. For example, in the fall some eleventh graders were j 
involved in athletics and in the spring one group of students was 
involved in a trip to Africa. Eleventh graders were attracted to 
the spring session because of the internship program. 

STUDENT OUTCOMES 

Scientific Inquiry - critical thinking ; A major objective of the 
study was to assess the progress and performance of students with 
regard to (Al) scientific inquiry or critical thinking (A2) self- 
esteem, and (A3) standardized tests. As indicated earlier, the 
Raven's Advanced Progressive Matrices were used to measure 

critical thinking. The matrices presented the students with 36 j 
problems in the form of a pattern with a piece taken out of it. 
The task was to select the missing piece from eight choices that 
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were presented below the pattern (see Figure I for a sample 
problem) . The problems from 1 to 36 become progressively more 
difficult, hence the name progressive matrices. Two types of 
results are reported here. First, the test is normed on the 
basis of age (12 tc 20) , hence the total number of correct 
answers was one basis for analyzing the results and determining 
percentile rank. 

Seventeen seventh graders tool the test and eleven scored in 
the 90-95th percentile. Six of the fifteen ninth graders taking 
the test scored in the 90-95th percentile and eight of the 
sixteen eleventh graders scored in the 9 0-95th percentile. 
Overall, 25 of the 45 students or 57 percent scored in the 90- 
95ta percentile range (Table IV) , 

TABLE IV 

PERCENTILE SCORES OF STEP STUDENTS ON THE RAVEN'S 





BELOW 
50% 


50- 


-74% 


75 


-89% 


90- 


-95% 


TOTAL 
N 




f 


% 


# 


% 


# 


% 




% 




GRADE 11 


2 


15 


2 


15 


1 


8 


8 


62 


13 


GRADE 9 


1 


8 


1 


8 


5 


38 


6 


46 


13 


GRADE 7 


1 


6 


1 


6 


4 


24 


11 


65 


17 


TOTALS 


4 


9 


4 


9 


10 


23 


25 


58 


43 
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That .neans that nearly half of the 52 STEP students demonstrated 
excellent observation and clear thinking skills. Although these 
tests were administered alter the program began, it cannot be 
determined at this juncture if there was any program impact. 
However, it is clear that these students, in general, have 
ability to do scientific studies. 

To determine a more in depth understanding of the 
performance of students on the Raven's test, we analyzed the 
common errors and error type by grade. 

Four types of errors can be identified: 

A. Incomplete solutions. These were errors due to people 
failing to grasp all the variables determining the nature of 
the correct figure required to complete a test item. 
Instead, they chose a figure which was right as far as it 
went but was only partly correct. 

B. Arbitrary lines of reasoning. Here the figure chosen 
suggests that the person has used a principle of reasoning 
qualitatively different from that demanded by the problem. 

C. Over-determined choices. These were errors involving 
failure to discriminate irrelevant qualities in the figure 
chosen, and to choose a figure which combined as many as 
possible of the individual characters shown in the matrix to 
be completed. 

21 
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D. Repetitions. These are errors made by people who simply 

selected a figure identical with one of the thr^e figures in 
the matrix immediately adjacent to the space to be filled. 



Tables V, VI, and VII show the position and nature of the 
two most common errors made at three levels of difficulty. Since 
our sample of students taking the test was small (48) , we were 
not able to determine the most common error for some items, nor 
were we able to determine the second most common error for some 
items. 

For each grade level however, we were able to determine (1) 
the number and percentage of students who chose the correct 
figure to solve each problem and (2) the frequency with which 
each of the seven erroneous figures were chosen. Errors due to 
failure to attend to all relevant aspects of the problem i.e. 
incomplete correlate errors, are clearly the most frequent across 
all three grade levels, accounting for more than a third of the 
erroneous selections made at all levels of difficulty. 

Errors due to applying the wrong principle to the solution 
of a problem were, if anything most frequent at the lowest level 
of difficulty but probably not significantly so. Errors due to 
confluence of ideas and repetition, not very common in any case, 
declined in frequency of occurrence as the total score increased. 
The errors of our small sample of 48 students (17 seventh 
graders, 15 ninth graders and 16 eleventh graders) are almost 
identical to the larger normed sample reported by Raven's (1983) 
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in earlier studies, since the reliability (,91) is relatively 
high on these untimed tests, we are confident that the 
identification of errors for our sample is correct • 
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TABLE V 



A 
B 



Position and Nature of Commonest Errors Made 
at Three Levels of Ability 
For Seventh Grade STEP Students (1988) 
Incomplete Correlates C - Confluence of Ideals 

Wrong Principle D - Repetition 



xTODlGIn 


Percentage 


Correct 


Commonest 


error 


Second 


no . 


passes 


figures 






commonest 


error 








Position Nature 


Position 


Nature 


LEVEL) 1 














1 


76 


5 




- 






2 


65 


1 


2 


D 






3 


88 


7 


3/4 . 


B 


- 




A 

4 


7^ 


4 


4 


— 


3/8 


B 


R 


oo 
oo 


J 


1/4 


A 


- 




o 


£R 


1 


7 


D 


4/2 


D 


7 




6 


2 


C 


5 




Q 

O 


A *7 

47 


1 


2 


— 


5 




Q 


RO 


8 


1 


— 


3/4 


D 


10 


53 


4 


8 


B 


1 




1 1 
11 


65 


5 


3 
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TABLE VI 

Position and Nature of Commonest Errors Made 
at Three Levels of Ability 
For Ninth Grade STEP Students (1988) 
A - Incomplete Correlates C - Confluence of Ideals 

B - Wrong Principle D - Repetition 



Problem Percentage Correct Commonest error Second 

no . passes figures commonest error 

Position Nature Position Nature 

uEVEL 1 



1 


67 


5 


1/2 


AA 


3 


2 


80 


1 








3 


87 


7 


8 






4 


73 


4 








5 


73 


3 


1 


A 


2 


6 


87 


1 


2/7 


DD 




7 


67 


6 


2 


C 


7/8 


8 


47 


1 


3 




2/8 


9 


93 


8 


4 


B 




10 


60 


4 


8 


B 


2/3 


11 


87 


5 


7/1 


A 




12 


53 
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5 


B 


7 



LEVEL 2 
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5/6 


A 




14 


67 
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8 
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15 


47 
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8 
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CB 
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17 
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7 
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We are somewhat constrained from a more detailed analysis 
due to the small sample. Our hope is to eventually create norms 
for the STEP sample as the number of students increase. However 
our analysis of common errors makes t possible to suggest to 
teachers and the instructional staff of STEP that more attention 
be paid to providing experiences where students are asked to 
identify all of the key variables in solving a problem. This is 
what is meant by incomplete correlates. To put this another way, 
students need to make sure that they have all of the relevant 
information before identifying the solution to a problem. 

A second observation is that students need to be taught 
principles of reasoning and logic in order to avoid making 
arbitrary judgments when seeking to find a solution to a problem. 
It is difficult, if not impossible to solve problems consistently 
if one's logic is incorrect. There are a range of materials and 
exercises that help students develop and understand deductive or 
inductive reasoning. 

Unfortunately, logic is not a topic that is taught very much 
in most schools but we believe that it is the basis for improving 
critical thinking skills for adolescents like those in the STEP 
program. 

Standardized Test Results : At this juncture we have a report on 
eleventh grade PSAT scores. Apparently these students did not do 
well on the PSAT given in October of 1988. There were only four 
students who scored 500 or above, two in math and two on the 
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verbal section, while five students scored between 440 and 499. 
Nevertheless, this represented an improvement over a pre-PSAT 
given in August (see Table VIII) . Students taking the test both 
times averaged 370 in August and 430 in October, a gain of 60 
points. It is difficult to determine if these gains were the 
result of the STEP program, but the scores clearly indicate a 
move in a positive direction. 
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TABLE VIII 



PSAT RESULTS - 11TH GRADE STEP 



I . D. 


Math 8/88 


MATH Oct 88 


VERBAL Oct 88 


102 


350 


410 


410 


105 


470 


480 


500 


116 


570 


490 


450 


109 


530 


590 


510 


106 


300 


360 


320 


113 


360 


460 


350 


111 


410 


N.A. 


N.A. 


113 


N.A. 


420 


400 


104 


200 


340 


250 


110 


390 


N.A. 


N.A. 


122 


N.A. 


320 


340 


103 


270 


400 


360 


107 


320 


430 


520 


115 


290 


320 


270 


112 


320 


420 


280 


119 


N.A. 


370 


370 


114 


390 


360 


350 


101 


480 


570 


360 


120 


N.A. 


350 


540 


121 


N.A. 


450 


380 


Nat'l 


College Norm 


450 
(8 below) 


410-420 
fll below this #) 
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In summary, the performance of STEP students on the PSAT 
compared to their critical thinking ability as indicated by the 
Raven's Matrices suggests that their PSAT is not commensurate 
wi':h their ability. It is too early and in any case difficult to 
determine reasons for this discrepancy. Some would argue that 
the PSAT and SAT tests are not valid measure of academic ability 
(Owen, 1989). Clearly steps need to be taken to take advantage 
of the critical thinking abilities of these students. We suspect 
that the STEP program, for example, needs to present greater 
challenges to students in the instructional modules, in any 
case, there is an overriding need for the program. 

Self-Esteem and Achieveme nt Responsibility : The Rosenberg self- 
esteem scale asks students to (A) strongly agree (B) Agree (C) 
Disagree or (D) Strongly disagree with the following statements. 
Note that the statements are stated both positively (# 1,2,4,6, 
and 7) and negatively (# 3,5,8,9, and 10). 

1. I feel that I'm a person of worth, at least on an equal 
plane with others. 

2. I feel that I have a number of good qualities. 

3. All in all, I am inclined to feel that I am a failure. 

4. I am able to do things as well as most other people. 

5. I feel I do not have much to be proud of. 

6. I take a positive attitude toward myself. 

7. On the whole, I am satisfied with myself. 

8. I wish I could have more respect for myself. 
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9. I certainly feel useless at times. 

10. At times I think I am no good at all. 

The results of the scores on the Rosenberg scale of self- 
esteem revealed in general that STEP students felt good about 
themselves. The overall positive scores on the ten-item scale 
averaged above 90 percent. 

An item analysis of this scale is underway and will probably 
provide better clues about students' attitudes toward themselves 
and schools that can be useful in assisting students in coping 
with the educational environment. We are also examining the use 
of more qualitative methodology to examine self-esteem. 

The Brcokover scale also uses a five-point scale ranging 
from positive to negative on the following questions: 

1. How do you rate yourself in school ability compared to your 
close friends? 

2. How do you rate yourself in school ability compared to those 
in your class at school? 

3. Where do you think you would rank in your class in college? 

4. Do you think you have the ability to compete in college? 

5. Where do you think you would rank in your class in college? 

6. In order to become a doctor, lawyer, or university 
professor, work beyond four years of college is necessary. 
How likely do you think it is that you could complete such 
ar" anced work? 

7. Forget for a moment how others grade your work. In your own 
opinion, how good do you think your work is? 

8. What kind of grades do you think you are capable of getting? 
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On the surface, student responses to the Brookover scale of 
achievement responsibility appeared strongly positive. However, 
a preliminary item analysis suggests that students have doubts 
about their ability to succeed in academic areas. Nevertheless, 
they do believe they are capable and see STEP as an avenue toward 
success. 
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SUMMARY AND RECOMMENDATIONS 
The report presented here is part of a longer more detailed 
examination of the Manhattanville-Woodlands STEP program (Miller 
and Repa, 1989). Some of the supporting documents, tables and 
charts have been omitted. The longitudinal nature of the study 
and the fact that the study is formative as well as summative 
means that all of our reports are continuous. Nevertheless we 
are able to draw some conclusions regarding various aspects of 
the program. 

In regard to the STEP instructional program, there appear to 
be two major components that can be improved. First is the 
modules, in terms of (1) curriculum content and (2) critical 
thinking skills. As we suggested in the section on results, both 
Woodlands teachers and STEP instructors need to provide students 
with learning experiences that need to be taught principles of 
reasoning and logic in order to avoid making arbitrary judgments 
when seeking solutions so a problem. A closer alignment between 
math and science content in the instructional modules and school 
math and science content areas would help the students improve in 
both areas. Both the high school teachers and the STEP 
instructors need to focus on critical thinking skills. A staff 
development workshop might be very useful. The standardized test 
scores of STEP students do not appear to be commensurate with 
their critical thinking skills and students need to be presented 
with greater challenges. 
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